Homogeneity bias in Large Language Models (LLMs) refers to their tendency to homogenize the representations of some groups compared to others. Previous studies documenting this bias have predominantly used encoder models, which may have inadvertently introduced biases. To address this limitation, we prompted GPT-4 to generate single word/expression completions associated with 18 situation cues - specific, measurable elements of environments that influence how individuals perceive situations and compared the variability of these completions using probability of differentiation. This approach directly assessed homogeneity bias from the model's outputs, bypassing encoder models. Across five studies, we find that homogeneity bias is highly volatile across situation cues and writing prompts, suggesting that the bias observed in past work may reflect those within encoder models rather than LLMs. Furthermore, these results suggest that homogeneity bias in LLMs is brittle, as even minor and arbitrary changes in prompts can significantly alter the expression of biases. Future work should further explore how variations in syntactic features and topic choices in longer text generations influence homogeneity bias in LLMs.

Large Language Models (LLMs) have become pervasive in everyday life, yet their inner workings remain opaque. While scholarly efforts have demonstrated LLMs' propensity to reproduce biases in their training data, they have primarily focused on the association of social groups with stereotypic attributes. In this paper, we extend this line of inquiry to investigate a bias akin to the social-psychological phenomenon where socially dominant groups are perceived to be less homogeneous than socially subordinate groups as it is reproduced by LLMs. We had ChatGPT, a state-of-the-art LLM, generate a diversity of texts about intersectional group identities and compared text homogeneity. We consistently find that LLMs portray African, Asian, and Hispanic Americans as more homogeneous than White Americans. They also portray women as more homogeneous than men, but these differences are small. Finally, we find that the effect of gender differs across racial/ethnic groups such that the effect of gender is consistent within African and Hispanic Americans but not within Asian and White Americans. We speculate possible sources of this bias in LLMs and posit that the bias has the potential to amplify biases in future LLM training and to reinforce stereotypes.

Stephon Sanders is a soft-spoken 16-year-old Black student in Tampa, Florida who loves playing basketball and video games like Fortnite and Apex Legends and can often be found wearing an athletic jacket and basketball shorts. He's also the founder and chief executive officer of a mobile entertainment business, Street Gamez, that can amp up any party with a 32-foot, $55,000 trailer full of video game consoles for up to 28 players. He plans to add a $45,000 mobile gaming bus, staffed by a second crew, and move beyond the Tampa area to serve North and South Carolina. Sanders' single mother, Tiffany-Autumn Bell, 35, cashed out much of her savings and gave up nearly all of her already scarce free time to support her then-13-year-old son's dream of launching a mobile entertainment video business. To Bell, an Army veteran who served in Afghanistan and has her own full-time job as an IT project manager, helping Sanders become an entrepreneur was about making sure his life and career options were not limited by his race.

English is spoken by hundreds of millions of people worldwide, but do the development of translation technology and "hybrid" languages threaten its status? Which country boasts the most English speakers, or people learning to speak English? According to a study published by Cambridge University Press, up to 350 million people there have at least some knowledge of English - and at least another 100 million in India. There are probably more people in China who speak English as a second language than there are Americans who speak it as their first. But for how much longer will English qualify as the "world's favourite language"?